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LEAST- INDEX  RESOLUTION  OE  DEGENERACY  IN  QUADRATIC  PROGRAMMING 


by 

Yow-Yieh  CHANG  and  Richard  W.  COTTLE 
Department  of  Operations  Research 
Stanford  University 


ABSTRACT 

In  this  study,  we  combine  least-index  pivot  selection  rules  with 
Keller's  algorithms  for  quadratic  programming  to  obtain  a finite  method 
for  processing  degenerate  problems. 


1 . Introduction . 

Degeneracy  is  a theoretically  troublesome  phenomenon  in  the  analysis 
of  simplicial  methods  for  linear  programming  although,  according  to 
folklore,  it  is  not  a serious  impediment  to  practical  computation  [7,  p.  2311. 
For  the  record,  a basic  solution  to  a linear  programming  problem  is  said 
[7,  p.  8 1 1 to  be  degenerate  if  at  least  one  of  the  basic  (i.e.  dependent) 
variables  equals  0.  Even  if  the  initial  basic  solution  is  non-degenerate, 
an  adjacent  extreme  point  algorithm  such  as  Dantzig's  simplex  method  may 
select  degenerate  basic  solutions  as  a consequence  of  ties  in  the 
minimum  ratio  test  used  to  determine  the  exiting  basic  variable.  This  in 
turn  can  lead  to  the  phenomenon  known  as  circ 1 ing  (alias  cyling) : a 
sequence  of  bases  which  (after  finitely  many  steps)  repeats  itself.  De- 
generacy per  se  is  not  the  problem;  but  when  it  is  present,  circling  is 
a possibility  and  must  be  avoided  if  the  simplex  method  and  procedures  like 
It  are  to  work  and  be  finite.  The  "degeneracy  problem"  refers  to  the 
difficulties  associated  with  circling. 


Until  quite  recently,  the  theoretical  procedures  available  for 
handling  the  degeneracy  problem  were  Chames'  perturbation  technique  | 3 1 , 
the  lexicographic  method  of  Dantzig,  Orden  and  Wolfe  | 8 ] , the  ad  hoc  method 
of  Wolfe  | 15] . We  shall  not  attempt  to  review  these  technical  procedures 
here,  but  simply  remark  that  their  actual  implementations  are  commonly  be- 
lieved to  entail  storage  requirements  and  computational  effort  out  of  all 
proportion  to  the  need  for  them.  In  1976,  Bland  ( 2 ] announced  a finite 
version  of  the  simplex  method  based  upon  a double  least-index  pivot  selection 
rule.  In  Bland's  method,  the  entering  basic  variable  is  chosen  as  the 
candidate  with  the  smallest  index.  The  variable  it  replaces  (if  any)  is 
determined  by  the  usual  minimum  ratio  test  with  ties  broken  according  to 
the  least-index  rule.  The  method  seems  so  natural,  it  is  difficult  to 
understand  why  it  was  not  published  earlier. 

At  this  point,  Bland's  contribution  is  of  greater  importance  to  linear 
programming  theory  than  practice.  A study  by  Avis  and  Chvatal  | 1 ] shows 
that  the  method  (in  the  form  sketched  above)  is  less  efficient  than  those 
which  choose  the  entering  variable  by  paying  more  attention  to  decreasing  the 
objective  function.  Avis  and  Chvatal  found,  however,  that  prior  arrangement 
of  the  variables  may  help  to  bring  about  greater  efficiency  as  compared  with 
the  other  methods.  Another  possible  shortcoming  of  Bland's  method  may  lie 
in  its  disregard  of  the  magnitude  of  the  pivot  entry.  Recognizing  these 
objections,  we  regard  Bland's  least-index  rule  as  a significant  contribution 
to  LP  theory  and  hope  that  it  will  develop  into  a viable  practical  approach 


as  well 


In  this  paper  we  demonstrate  that  Bland's  least-index  pivot  selection 
rules  can  be  applied  to  handle  the  degeneracy  problem  In  the  Dantzlg/van  de 
Panne-Whins ton/ Keller  algorithm  for  quadratic  programming.  (See  (6  1,  (7), 
I 13) , [9  1.)  In  a nutshell,  what  Is  shown  is  that  by  using  Bland's  least- 
index  rules  in  Keller's  algorithm,  one  can  prevent  circling  and  reach  one 
of  the  terminal  forms  after  finitely  many  pivot  steps.  From  this  one 


either  has  knowledge  that  the  objective  function  is  unbounded  below  or  else 


a solution  of  the  Kuhn-Tucker  conditions 


1 


I 

! 
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2 . Quadratic  programming 

In  this  section  we  consider  the  application  of  Bland's  least  index 
rule  to  the  symmetric  Dantzig/van  de  Panne-Whinston/Keller  quadratic 
programming  algorithm.  This  procedure  in  its  most  general  form  (given 
by  Keller)  aims  to  find  a local  minimum  in  a nonconvex  quadratic  program- 
ming problem. 


2.1  Statement  of  the  problem. 

We  express  the  quadratic  programming  problem  in  the  form 

(1)  minimize  Q(x)  ■ c^x  + \>xTDx 
subject  to  Ax  < b 

x > 0 

The  matrix  D is  always  (without  loss  of  generality)  assumed  symmetric. 

When  we  speak  of  convex  quadratic  programming,  we  refer  to  the  case  where 
D is  positive  serai-definite;  but  for  the  moment  this  is  not  assumed. 

The  approach  taken  in  the  Dantzig/van  de  Panne-Whinston/Keller 
algorithms  for  solving  (1)  is  to  find  a solution  of  its  Kuhn-Tucker  con- 
ditions, i.e.  its  necessary  conditions  of  optimality,  namely 

(2)  u ■ c + Dx  + y 
v » b - Ax 

x>0,y>0,u>0,v>0 

T T 

x u ■ 0 , y v ■ 0 

In  the  convex  quadratic  programming  case  these  conditions  are  also  sufficient 


\ 
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r 


! 
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for  the  optimality  of  x.  Tn  the  nonconvex  case,  they  arc  not  in  general 
sufficient,  even  for  a local  minimum. 

The  method  of  Keller  (which  extends  those  of  Dan trig  and  van  de  Panne- 
Whinston)  is  capable  of  finding  a local  minimum  of  a nondegenerate  non- 
convex  QP  in  finitely  many  steps.  This  is  done  by  obtaining  a suitable 
solution  of  (2).  We  show  here  that  by  using  a natural  adaptation  of  Hlnnd's 
least- index  rule  we  can  dispense  with  the  nondegeneracy  assumption  and  find 
a solution  of  (2),  again  in  finitely  many  steps. 


\) 

* 
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2.2  Bland's  least-index  rule  for  l.P. 

In  197b,  R.G.  Bland  | 2 1 showed  that  a certain  simple  and  natural 
pivot  selection  rule  never  leads  to  circling.  His  pivoting  rule  is  a 
refinement  of  the  simplex  rule  obtained  bv  Imposing  the  following  restrie- 
t ions: 

(a)  among  all  the  candidates  to  enter  the  basic  set  of  variables, 
select  the  nonbasic  variable  having  the  lowest  index. 

(b)  among  *11  candidates  to  leave  the  basic  set,  select  the  basic 
variable  having  the  lowest  Index. 

It  is  well  to  understand  what  is  meant  by  these  statements.  We 
imagine  the  linear  program  to  be  expressed  In  the  canonical  simplex  tableau 
consisting  of  m + 1 linearly  independant  rows  and  n + 2 columns: 


variables 

xo 

X1 

• • • X 

n 
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m 

0 
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• • • a 

mn 

b 

m 
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N - ■ i-'-' > ■ v ^gp 


It  Is  assumed  that  BQ  - 0 , 1 < < n for  i - 1,  ....  m , and  the 

matrix  formed  from  the  columns  headed  bv  xu  , x„  « x constitutes 

o,.  B,  n 

0 1 m 

an  identity  matrix  of  order  (m  + 1).  In  particular,  the  columns 

associated  with  x has  a 1 in  row  1 and  0 elsewhere. 

Bi 

The  least-index  selection  rule  for  choosing  the  incoming  variable  is 
clear  although  it  differs  from  the  customary  one.  We  want  to  stress  that  in 
choosing  the  exiting  basic  variable  one  uses  the  usual  candidacy  rule  and 
breaks  ties  according  to  the  least  index  of  the  affected  basic  variables 
rather  than  by  the  least  row  number  of  such  variables.  To  be  precise, 
suppose  x^  is  the  incoming  variable.  Then  the  exiting  basic  variable 
(If  any)  is  the  one  whose  index  (subscript)  satisfies 

(3)  B - min 

r 

The  pivot  element  is  then  a . The  important  point  is  that  the  minimization 

r s 

in  (3)  is  over  B^,  rather  than  C. 


» 0 


and 


min 
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i_ 

is 


is 
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2 . 3 Keller's  algorithm. 

In  describing  Keller's  algorithm.  It  is  convenient  to  introduce  a change 
of  notation  in  (2).  Suppose  A £ Rmxn,  Initially,  we  define  I ■ il,  ....  n ) , 
J ■ {n  + 1,  ....  n + ml  , and  then  put 

(4)  x7  - x . x , - v , y7  - u , y r - y 


6 


Accordingly,  (2)  can  be  replaced  by  the  bisymmetric  schema: 


1 xr  V 


T T 
0 c -b 


D A 
-A  0 


The  variable  6 is  related  to  the  objective  function  through  the 
equation 


6 - 2Q  - xly.  - x^y. 

A i «/  t' 


This  equation  is  a consequence  of  two  other  equations  represented  in  (5). 
Actually,  Keller's  method  works  with  the  slightly  more  general  schema 


(7) 

1 

xr 

V 

0 - 

2k 

T 

c 

-bT 

c 

D 

AT 

X 7 - 

V 

b 

-A 

E 

in  which  E is  sywnetr lc  and  positive  seml-def inlte  and  ic  Is  a constant. 

(In  our  original  schema  (5)  we  have  E - 0 and  *c  - 0.)  The  most  important 
reason  for  working  with  such  a schema  Is  that  one  encounters  this  type  of 
format  in  subsequent  principal  pivotal  transforms  of  (5>,  so  one  might  as 
well  regard  it  as  given  from  the  start.  A second  rationale  for  (7)  is  that 


7 


it  corresponds  to  the  one  used  for  symmetric  dual  quadratic  programs  l 4] 


In  such  a schema,  we  regard  the  variables  y^  (i  6 I)  and  Xj  (J  G J)  as 
basic . The  variables  x^  (i  G I)  and  y^  (J  G J)  are  then  non-basic. 

It  is  helpful  to  notice  that  the  entries  in  the  schema  can  be  considered  as 
partial  derivatives.  For  example 


3y1/3xj 


‘ij 


It  is  assumed  that  b > 0.  This  amounts  to  a (primal)  feasibility 
condition.  In  a given  instance,  if  b is  not  nonnegative  one  can  as  an 
initialization  step  enecute  the  Phase  I procedure  of  LP  to  make  it  so  or 
else  determine  the  infeasibility  of  the  constraints.  The  remaining  steps 
of  the  original  method  for  the  nondegenerate  case  run  as  follows: 

Step  1.  If  all  basic  y-variables  are  nonnegative,  stop.  A local  minimum  has 
has  been  found.  Otherwise  choose  a negative  y-component , say  y , as  the 
"distinguished  variable". 

Step  2.  If  3yg/3xs  £ 0 go  to  Step  3.  Otherwise,  determine  the  "blocking 

variable",  i.e.  the  basic  variable  (either  y or  x.,  j G ,r)  which  reaches 

8 J 

the  value  0 first  as  x is  increased  from  0. 

s 

a.  If  y is  the  blocking  variable,  perform  the  in-pivot  (v  , x ) 

s — — s s 

by  which  y and  xs  are  exchanged.  Replace  J by  J U {s}  and  I by 
I - (s).  Return  to  Step  1. 

b.  If  xt  is  the  blocking  variable  and  Sx^/Sy^  > 0 then  perform  the 
out-pivot  <x^,  yt>  by  which  xf  and  y^  are  exchanged.  Replace  J by 

J - { t } and  J by  JU  (t).  Repeat  Step  2 (with  x^  still  the  driving 


8 


variable).  If  3x  /3y  = 0,  then  perform  the  exchange  pivot  (x  , x ), 

t t * c s 

(ys.  y ) through  which  xf  and  x^  are  exchanged  and  yg  and  y£  are 


exchanged.  Replace  J by  (*7  — {t))kJ{s}  and  I by  (f  - (si)  U{t}. 

Return  to  Step  1. 

Step  3.  Determine  whether  x is  blocked.  If  ?x,/3x  > 0 for  all 

1 s j s ■* 

j € J then  x is  unblocked.  Stop,  since  Q “*  Otherwise  x is 

s s 

blocked.  If  Sx^/^y^  > 0,  perform  tiie  out-pivot  ( x^ , y£ ) ; replace  J 
by  J - {t)  and  I by  I U { t } . Return  to  Step  2.  If  3xt/3yt  ■ 0, 
perform  the  exchange  pivot  (x^,  xs ^ » (ys»  yt ) ; replace  J by 
(«/-{t})U{s)  and  I by  (I  - [9})  U { t } . Return  to  Step  1. 

We  do  not  intend  to  justify  the  method  or  even  discuss  it  at  length. 

For  this  one  should  see  1 9).  However,  we  do  wish  to  draw  the  reader's 
attention  to  some  of  its  salient  features. 

1.  The  nondegeneracv  assumption  implies  that  there  is  always  at  most  one 
blocking  variable. 

2.  The  method  uses  only  principal  pivots  of  order  1 (in-pivots  and  out- 
pivots)  or  order  2 (exchange  pivots). 

3.  The  property  of  bisymmetry  is  preserved  by  principal  pivoting  (regard- 
less of  the  order).  This  is  proved  in  (14  ] . 

4.  The  nonnegativity  of  the  x-variables  is  preserved  throughout  the 
procedure. 

5.  Given  the  current  index  set  / the  principal  submatrix  E corresponding 
to  the  rows  and  columns  indexed  by  J is  always  positive  semi-definite 
and  its  nullity  equals  that  of  the  original,  E. 

6.  According  to  quadratic  programming  theory,  a Kuhn-Tucker  point  is  a local 


L 
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minimum  _in  the  nondegenerate  case  provided  E is  positive  semi- 
definite.  However,  in  the  degenerate  case,  more  than  just  the  positive 
semi-definiteness  of  E is  needed.  In  effect,  a type  oi  copositivity 
condition  is  required.  (See  [10]  and  [11]).  Unfortunately  Keller's 
method  makes  no  provision  for  this;  hence  we  make  no  claim  tha*.  the 
modified  Keller  method  described  below  actually  yields  a local  minimum 
in  the  degenerate  case.  Nevertheless,  the  method  leads  in  finitely  many 
steps  to  an  indication  of  unboundedness  or  else  to  a solution  of  the 
Kuhn-Tucker  conditions,  and  in  the  case  of  convex  quadratic  programming, 
this  is  enough  for  global  optimality. 

2 . 4 Finiteness  of  Keller's  method  with  the  least  index  rule . 

We  now  give  a modification  of  Keller's  algorithm  and  prove  its 
finiteness  without  recourse  to  a nondegeneracy  assumption.  To  accomplish 
this,  we  introduce  a refinement  of  Keller's  algorithm  which  imposes  the 
following  double  least- index  rule: 

(i)  In  Step  1,  choose  the  distinguished  variable  yg  so  that 
s » min  (i  6 I:  y^  < 0} 


(ii)  In  Steps  2(b)  and  3,  choose  the  blocking  variable  xt  so  that 


t ■ min  \ j G J : -a . < 0 and 

I J8 


min 


: k £ -a 


js 


ks 


ks  < °! 
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there  could  be  i "tie"  between 


blocking.  The  statement  of  Step  2 Is  intended  to  mean  that  under 


the  blocking  variable 


these  circumstances,  one  should  choose 


This  would  decrease  the  objective  function  value 


In  Keller's  method,  each  return  to  Step  1 completes  a major  cycle 


Lemma  1.  Each  major  cycle  of  Keller's  algorithm  (with  or  without  the 


least  index  rule)  is  finite 


Proof.  The  pivots  in  Keller's  method  are  of  three  types:  in-pivots 


Both  In-pivots  and  exchange-pivots  lead 


out-pivots,  and  exchange-pivot 


back  to  Step  1,  hence  each  completes  a major  cycle.  Each  out-pivot 


reduces  the  cardinality  of  ./  by  1,  so  there  can  be  only  finitely  many 


out-pivots  within  a major  cycle 


Lemma  2.  If  circling  occurs  in  Keller's  algorithm,  then  during  circling 


only  exchange-pivots  are  used 


Proof.  By  Lemma  1,  if  circling  occurs,  tin  re  must  be  infinitely  many 


returns  to  the  same  complementary  basis  in  terms  of  which  the  schema  is 


uniquely  determined.  Keller  shows  that  regardless  of  whether  degeneracy 


is  present,  the  value  of  9 is  nonincreasing  under  all  steps  of  the 


algorithm*  Hence  during  circling,  0 must  be  fixed.  This  implies 


there  can  be  no  in-plvots  during  circling,  for  these  always  decrease  the 


value  of  9.  Now  it  remains  to  show  that  during  circling,  there  are  no 


out-pivots.  To  see  this  we  note  that  each  out-plvot  increases  the 


cardinality  of  the  index  set  T (corresponding  to  the  basic  y-variables) 


1 


Since  exchange-pivots  do  not  affect  the  cardinality  of  I and  ln-plvots 


cannot  occur  by  our  previous  argument,  there  can  be  no  out-pivots 
during  circling.  □ 

Suppose  circling  occurs  during  the  execution  of  Keller's  algorithm. 


We  know  that  each  major  cycle  must  be  finite,  so  the  only  remaining 
possibility  is  that  there  are  infinitely  many  major  cycles  (returns 


to  Step  1).  A circle  begins  at  a complementary  schema,  a principal 
pivotal  transform  of  the  original  schema  (7).  Since  non-terminal 
principal  transforms  of  (7)  share  its  properties  (bisysmmetry , positive 
semi-definiteness  of  E,  b > 0,  c | 0),  we  may  assume  for  simplicity 
that  the  circling  starts  at  (7).  The  circling  consists  of  a sequence 
of  (at  least  two)  exchange  pivots  returning  to  (7).  Assuming  the 
circling  starts  at  (7)  we  focus  attention  on  the  variables  actually 
exchanged . 


Definition.  Let 


K ■ (J  € «7 : Xj  becomes  nonbasic  during  circling} 


Accordingly,  by  deleting  from  (7)  those  rows  and  columns  j such  that 
J we  obtain  a subschema  with  much  the  same  character,  namely 


1 *7  yl 


T T 
2‘  c < 


c D 


b„  -A  E 
K I K‘  KK 


Caution: 


In  (8)  and  the  lemma  below,  we  indulge  in  an  abuse  of 


notation.  The  symbols  b^  , A^  , and  E^  refer  to  those  parts  of 
b.  A,  and  E corresponding  to  the  basic  variables  x^. 

Lemma  3 . Under  the  assumptions  made  above, 

(9)  b^  - 0 and  E^  - 0 , - 0 

Proof.  Let  xg  be  the  first  driving  variable.  Since  this  must  lead 

to  an  exchange-pivot,  let  xt  be  the  blocking  variable.  Since  we  must 

perform  an  exchange-pivot  (rather  than  an  out-pivot)  we  must  have  e - 0. 

tt 

This  implies 


E 

t* 


0 . E.t  - 0 


because  E is  symmetric  and  positive  semi-definite.  As  a result  of  an 
exchange-pivot,  2k  becomes 


c , * b d 

* L*t.  \.  v ■ J\ 


2k  ♦ b 


Each  of  the  summands  within  the  square  brackets  is  negative.  Since 
b{  is  nonnegative  and  0 does  not  change  during  circling,  we  must  have 
*)t  " 0»  Under  these  circumstances,  the  entries  of  b^  and  E ^ are  not 
affected  by  the  exchange-pivot.  Since  each  row  indexed  by  k € K is 
Involved  in  an  exchange-pivot,  the  argument  Just  given  applies  and 
the  proof  la  complete.  D 


The  bounds  given  by  Marshall  and  Suurballe  are  sharp,  so  they  must 
be  sharp  here  too.  □ 


given  in  1 12 1 . 


lowing 

min  Ima  l 

examp 1 t 

Is  ; 

i modi fie 

X1 

X2 

X3 

X4 

y5 

-1 

7 

1 

2 

0 

-1 

0 

0 

0 

1/2 

0 

-1 

0 

0 

-11/2 

0 

0 

-l 

0 

-5/2 

0 

0 

0 

-1 

9 

-1/2 

11/2 

5/2 

-9 

0 

-1/2 

3/2 

1/2 

-1 

0 

After  6 pivots,  one  returns  to  the  same  schema. 


2.5.  On  dropping  degeneracy  precautions. 


As  is  well  known,  it  is  customary  In  practice  to  ignore  the 
degeneracy  problem  in  many  mathematical  programming  schemes.  Quadratic 
programming  is  no  exception.  Here  we  wish  to  draw  some  attention  to  the 
fact  that  occasionally  this  can  he  done  with  absolute  confidence.  To 
this  end,  we  consider  the  application  of  Keller's  algorithm  (minus 
degeneracy  precautions)  to  a special  type  of  problem. 


In  which  h > 0 and  F.  Is  positive  definite,  provided  J 1 <J>,  uses 
only  ln-pivots  and  out-pivots. 

Proof.  Suppose  J is  vacuous.  Let  y be  the  distinguished  variable; 

then  is  the  driving  variable.  Since  «7  - $,  y is  the  only 

eligible  blocking  variable.  If  d « 3y  /3x  < 0,  then  y does  not 

ss  s s - s 

block  x^.  The  procedure  terminates  with  t?  going  to  minus  infinity. 

If  d^s  > 0 then  the  in-pivot  (y^,  x8 ^ brings  about  a bisymmetric 
schema  with  J m {a}  , I • I - {»}  , and  E - H/d  1, 

Thus,  we  may  assume  J ♦ if*  Note  that  so  long  as  F.  remains 

positive  definite,  there  will  be  no  exchange  pivots,  for  these  come  about 

when  some  x_  blocks  an  x and  ?x  /3v  *■  e 0.  Now  clearly  an 

t s t yt  tt  J 

out-pivot  (x^,  yt>  preserves  the  positive  definiteness  of  F unless 
J " (t).  Under  the  action  of  an  in-pivot  <v  , x ) , E is  replaced 

' S 8 

by  a principal  transform  of  the  positive  definite  matrix 


Hence  E will  be  positive  definite  when  it  is  nonvacuous.  □ 


lb 


Corollary . Under  the  assumptions  of  Theorem  2,  Keller’s  method 

is  finite. 


Proof.  If  circling  occurs  In  Keller's  method  there  must  be  at  least 
two  exchange-pivots.  □ 

Remarks . Some  related  results  should  be  mentioned  in  connection  with 
Theorem  2.  In  his  Ph.D.  thesis  |10|,  Keller  notes  that  his  method 
executes  no  exchange-pivots  when  D Is  positive  semi-definite  and  E 
is  positive  definite,  but  the  discussion  there  is  not  related  to  the 
degeneracy  problem.  Also,  in  Appendix  I of  | 5 ) , Cottle  and  Djang 

observe  that  the  symmetric  van  de  Panne-Whlnston  algorithm  (13)  applied  , , 

I I 

to  the  least-distance  problem  studied  bv  Wolfe  | 1 1 does  not  require 
a non-degeneracy  assumption  to  assure  its  finiteness. 

We  have  presented  the  application  of  Bland's  double  least-index 
rule  to  Keller's  method  (and  therefore  implicitly  to  the  van  de  Panne- 
Whinston  symmetric  algorithm).  There  is  reason  to  believe  that  such 
a rule  could  apply  to  other  algorithms,  but  we  do  not  pursue  this 
possibility  here. 
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